{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Built-In Fully Connected Neural Network on MNIST \n",
    "\n",
    "In this notebook, we show how to train and evaluate a fully connected neural network on the MNIST classification problem using the Concrete ML library, our open-source privacy-preserving machine learning framework based on fully homomorphic encryption (FHE).\n",
    "\n",
    "Using a built-in model, this example emphasizes on the API's ease-of-use by illustrating the few main steps needed to create an efficient inference-secured Neural Network classifier. Thanks to the internal implementation of Quantize Aware Training (QAT) techniques, this Concrete ML `NeuralNetClassifier` model reaches a high accuracy score. More information about QAT is available in the [QAT notebook](QuantizationAwareTraining.ipynb).\n",
    "\n",
    "More precisely, the model is trained on clear data and its accuracy score is computed using FHE simulation, which is expected to be the same as if it was done in FHE. Then, the inference is executed in FHE on a few samples.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import libraries\n",
    "\n",
    "We import the required packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from concrete.compiler import check_gpu_available\n",
    "from joblib import Memory\n",
    "from sklearn.datasets import fetch_openml\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from torch import nn\n",
    "\n",
    "from concrete.ml.sklearn import NeuralNetClassifier\n",
    "\n",
    "use_gpu_if_available = False\n",
    "device = \"cuda\" if use_gpu_if_available and check_gpu_available() else \"cpu\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the data\n",
    "\n",
    "We download the train and test data-sets from OpenML. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "________________________________________________________________________________\n",
      "[Memory] Calling sklearn.datasets._openml.fetch_openml...\n",
      "fetch_openml('mnist_784')\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________fetch_openml - 26.7s, 0.4min\n"
     ]
    }
   ],
   "source": [
    "# scikit-learn's fetch_openml method doesn't handle local cache:\n",
    "# https://github.com/scikit-learn/scikit-learn/issues/18783#issuecomment-723471498\n",
    "# This is a workaround that prevents downloading the data every time the notebook is ran\n",
    "memory = Memory(\"./data/MNIST\")\n",
    "fetch_openml_cached = memory.cache(fetch_openml)\n",
    "\n",
    "# Fetch the MNIST data-set, with inputs already flattened\n",
    "mnist_dataset = fetch_openml_cached(\"mnist_784\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now need to normalize the values and split the inputs and targets into a test and train data-set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define max, mean and std values for the MNIST data-set\n",
    "max_value = 255\n",
    "mean = 0.1307\n",
    "std = 0.3081\n",
    "\n",
    "# Normalize the training data\n",
    "data = (mnist_dataset.data) / max_value\n",
    "data = ((data - mean) / std).round(decimals=4)\n",
    "\n",
    "# Concrete ML's NNs do not support: category, str, object types\n",
    "# FIXME: https://github.com/zama-ai/concrete-ml-internal/issues/2990\n",
    "target = mnist_dataset.target.astype(\"int\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_size = 10000\n",
    "x_train, x_test, y_train, y_test = train_test_split(\n",
    "    data, target, test_size=test_size, random_state=0\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot the first images from the train data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_samples(data, targets, n_samples=5, title=\"Train target\"):\n",
    "    # MNIST images are originally of shape 28x28 with grayscale values\n",
    "    samples_to_plot = np.array(data)[:n_samples].reshape((n_samples, 28, 28))\n",
    "\n",
    "    fig = plt.figure(figsize=(30, 30))\n",
    "\n",
    "    for i in range(n_samples):\n",
    "        subplot = fig.add_subplot(1, n_samples, i + 1)\n",
    "        subplot.set_title(f\"{title}: {np.array(targets)[i]}\", fontsize=15)\n",
    "        subplot.imshow(samples_to_plot[i], cmap=\"gray\", interpolation=\"nearest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAACUQAAAHUCAYAAAD4T7DAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABR1klEQVR4nO3de5iVdbk38HsQGEFhkDMkImqoecBEBfOEihwqE8W9pczAQ2ZhbSVTqTylyc5OZq9b2zsTKyWTBPKwebehQCp4INlmBxSUQBEIlOEk5/X+4evoCDPM/GYxz8wzn891zXXJWuu7fvc8rvrCcLtWSaFQKAQAAAAAAAAAAEAONMt6AAAAAAAAAAAAgGKxEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEEWDUlJSUquvfffdt+gz7LvvvlFSUlL056Vurr/++hq9JmbOnJn1qACNkg6mKm+//XaMHTs2Bg4cGD179ozWrVtH69at45BDDokrr7wyVqxYkfWIAI2aDmZnXn/99Tj//POje/fusfvuu0fv3r3juuuuiw0bNmQ9GkCjpoOpziOPPBLf+ta3YuDAgdGuXbsoKSmJAQMGZD0WQC7oYKqybNmyuOuuu+LMM8+MvffeO1q2bBnt2rWLk046Ke65554oFApZj0gj0zzrAeCDRo4cud1tTz75ZCxYsCD69OkTRxxxRKX7OnbsWE+TFdf48ePj/PPPj+uuuy6uv/76rMcpqoULF0avXr3ipJNOiunTpxfteY844ogdvj4iIt544434wx/+EK1bt44jjzyyaGcCNCU6uPHbVR38xhtvxL//+79H+/bt45BDDoljjz021qxZE88//3x8//vfj3vvvTeefPLJ6NWrV9HOBGhKdHDjt6s6OCJi/vz5ceyxx8aKFSvi0EMPjRNOOCGef/75+M53vhPTpk2LadOmRWlpaVHPBGgqdHDjtys7+Nxzz43y8vKiPicA79LBjd+u6uCvf/3rce+990bz5s3jqKOOiuOPPz7eeOONePLJJ2PmzJnx8MMPx29+85vYbbfdinYm+WYhigZl/Pjx2902atSoWLBgQQwbNqxeymLatGmxefPmXX4OtTNs2LAYNmzYDu+76qqr4g9/+EOceeaZseeee9bvYAA5oYOpSo8ePeL555+Pj3/849Gs2ftvMLthw4a4+OKL41e/+lV84xvfiIkTJ2Y4JUDjpYOpzqhRo2LFihXxta99LX7yk59ERMSWLVviX//1X2PSpEkxbty43P1gHaC+6GCqM3z48Dj44IPjqKOOis2bN8egQYOyHgkgN3QwVenQoUN897vfjS9+8YvRqVOnitufe+65GDhwYEycODHuuuuuuPjiizOcksbER+bBh+y///5x0EEHZT0GNVQoFGLChAkREXHeeedlPA0AdaGDG6aysrLo27dvpWWoiIjdd989br755oiIePzxx7MYDYAi0cEN07PPPhtPPfVUdO7cOW655ZaK25s3bx533HFHtGjRIm677bbYsmVLhlMCUBc6uOG666674oorrogBAwZEmzZtsh4HgCLTwQ3TT37yk/jmN79ZaRkqIuLoo4+Oq6++OiKi4u+FoSYsRNFojR8/PkpKSuL666+Pl19+OUaMGBFdunSJZs2axeTJkyPi3beWv/766+PYY4+Nrl27RsuWLWPvvfeOL3zhC/Hyyy/v8Hl39JmxCxcurPiM8HfeeSeuvvrq6NmzZ5SWlsYBBxwQ3/ve92r8maUDBgyI888/PyIibrjhhkqfgfveRvR7Sz4jRoyI3r17xx577BFt2rSJY445Jv7jP/4jtm3btt3zXn/99RXP8eyzz8anP/3p6NChQ5SUlMTcuXMrnvc///M/o0+fPtGqVavo2rVrXHjhhbF8+fIYNWpUlJSU7PBtDd96660YO3ZsfOxjH4tWrVpFWVlZnHLKKfHwww9vN8N7H5czY8aMSt/bqFGjanR9amv69OmxePHi6Nq1awwcOHCXnAFAZTq4sqbawRERLVq0iIiIli1b7rIzAHifDq4s7x38yCOPRETE6aefvt3H4nXp0iVOOOGEePvtt+PJJ5+s0zkA7JwOrizvHQxAw6GDK2vKHdynT5+IiFiyZMkuO4P88ZF5NHrz5s2Lo48+Ojp06BAnn3xyvP322xV/Offzn/88brnlljj00EPj6KOPjtLS0vjrX/8av/rVr2LKlCnxxz/+MQ4//PAan7Vp06YYNGhQ/PWvf40BAwbEunXrYsaMGXH11VfHmjVr4qabbtrpcwwZMiS2bNkSTz311Hafg3vAAQdERMTGjRvjc5/7XHTo0CE+9rGPxZFHHhkrV66Mp59+OkaPHh3PPvvsDt9OMiJi5syZcfHFF0fv3r1j0KBBsWTJkop3dBgzZkzceuut0bJlyzj55JOjrKwsHn300Xj88cervA4vv/xyDBw4MBYvXhz77rtvDB48ONasWROzZ8+O008/Pb7//e/HFVdcERERRxxxRAwfPjx+97vfRZcuXWLIkCEVz3P88cdX/PN7n5lbjM+V/fWvfx0REZ/97Gd9XixAPdPBlTW1Dt68eXPF21d/6lOfqtNzAVA7OriyvHbw//7v/0ZExJFHHrnD+4888sh4/PHH48UXX4wBAwbU6DkBqBsdXFleOxiAhkcHV9YUO/jVV1+NiIiuXbvW+bloQgrQwI0cObIQEYXrrruu0u133313ISIKEVG49NJLC1u2bNkuO2vWrMKrr7663e2/+MUvChFROPnkk7e7r2fPnoUP/0/jtddeqzjrpJNOKpSXl1fc99xzzxV22223QuvWrQtr1qyp0ff03uwf/p7es3nz5sKkSZMKmzZtqnT78uXLC0cddVQhIgozZsyodN91111XMeP3vve97Z7zj3/8YyEiCu3bty/8+c9/rrh93bp1hcGDB1dkn3jiiYr7tmzZUjjssMMKEVG45ZZbClu3bq2475VXXin06tWrsNtuu1V6vveu1UknnbTT77+6x9TEO++8UygrKytEROFPf/pTnZ4LgO3p4Pfp4HddcMEFhZEjRxY+85nPFD7ykY8UIqJw3HHHFVasWJH0fADsmA5+X1Pu4I9//OOFiChMmTJlh/ffeuuthYgojBkzpsbPCUD1dPD7mnIHf9isWbOK8vNsAKqmg9+ng7e3adOmwsEHH1yIiMIPf/jDOj8fTYePzKPR69SpU3zve9/b4bsD9e/fv+It+z7o/PPPj+OOOy6mT58e5eXlNT6rWbNm8bOf/Szatm1bcdtRRx0VQ4cOjfXr18fzzz+f9k18SPPmzWPYsGEVm83v6dSpU4wbNy4iIqZMmbLD7GGHHRbf+MY3trv9zjvvjIiIyy+/PA499NCK21u3bh233XZbxdbwBz300EPx5z//OYYPHx7f+MY3Kj3mgAMOiB/+8IexdevW+K//+q9afX9lZWVx4IEHxj777FOr3If9/ve/j/Ly8jjkkEPi4x//eJ2eC4Da08GVNYUOvueee+Kee+6J3//+9/HGG2/EgAED4te//nV06NAh6fkASKODK8trB69du7Zi3h3ZY489IiJizZo1tZoFgHQ6uLK8djAADY8OrqypdfA111wTf/vb36JXr15xySWX1Pn5aDp8ZB6N3sCBA6v84WDEuz9AfOihh2Lu3Lnx1ltvxebNmyMi4s0334xCoRALFiyo8u3nP6xnz55x4IEHbnd77969K56zmObOnRv/8z//E//4xz9i/fr1USgUKn7Q+corr+ww8+lPf3q7z7yNiHjqqaciIuJf/uVftruvd+/eccQRR8Sf/vSnSrf/z//8T0REnHXWWTs864QTToiIiGeffbaG39G7zjzzzDjzzDNrldmR9z4u77zzzqvzcwFQezq4sqbQwVu2bImId6/3U089FWPHjo3DDjssJk6cGIMHD05+XgBqRwdX1hQ6GICGQQdXpoMBqC86uLKm1MG/+c1v4pZbbondd9897rvvvmpfB/BhFqJo9KrbKn388cdjxIgR8c9//rPKx9Tmv6Tce++9d3h7mzZtIuLdz3othk2bNsWoUaNiwoQJVT6mqrmruh7vlXOPHj2qzH24ABcuXBgREeeee26ce+65Vc6yYsWKKu/bVVauXBlTp06NZs2aVTsbALuODq6sqXRwRES3bt3i7LPPjqOPPjoOO+ywGDVqVMyfP7/inSoA2LV0cGV57eA999wzIiLWr1+/w/vXrVsXEe//uwBg19PBleW1gwFoeHRwZU2lgx9//PEYNWpUNGvWLCZMmBD9+/ev9xlo3CxE0ejtvvvuO7x97dq18a//+q/x1ltvxbXXXhsjRoyInj17RqtWraKkpCQ+97nPxYQJE6JQKNT4rB29jeCu8KMf/SgmTJgQhx12WNxyyy1x5JFHxl577RUtWrSIl19+OQ488MAq567qeqTYtm1bREQMGTIkunTpUuXjOnbsWLQza+r++++PzZs3x8knn1zlb0wA2LV0cGVNpYM/qGfPnnHCCSfEo48+Gs8880yccsopmc4D0FTo4Mry2sH77LNPvPDCC/H666/v8P73bu/Zs+cunwWAd+ngyvLawQA0PDq4sqbQwc8991ycccYZsWnTprjrrrti2LBh9Xo++WAhitz64x//GCtXroyzzz47brjhhu3uf/XVVzOYqmYmTZoUERETJkyIQw45pNJ9qXN369YtFi5cGIsXL97h2zwuXrx4u9veWzS66KKLYvjw4Unn7io+Lg+g4dLBleWtgz/svT8IV/dfYAFQP3RwZY29g/v06RNTpkzZ7r/efc97tx9++OH1ORYAO6CDK2vsHQxA46GDK8tLB//1r3+NoUOHxtq1a+PHP/5xnH/++VmPRCNVP+uNkIG33347Inb8tobz58+v8geK9aFly5YREbFly5Yd3l/d7L/97W+TzjzuuOMiIuJ3v/vddvfNnz8/Xnjhhe1uP+200yLi/UKuiZ19b8Xw6quvxqxZs6JVq1YNppgBeJ8OrixPHfxhW7dujSeffDIiIvbff/96OxeAHdPBlTX2Dv7Upz4VEREPPfTQdh/JsGzZsvjjH/8Ye+21V8X3CUB2dHBljb2DAWg8dHBleejghQsXxqBBg2LlypVx/fXXx2WXXVb0M2g6LESRW717946IiAcffLDSOxasWrUqLrzwwti8eXNWo0X37t0jImLevHk7vP+92e+8885Kt0+cODF++ctfJp35pS99KSLeffvFv/71rxW3v/POO/G1r32t4u0QP2j48OHxsY99LO6999648cYbt/sBbKFQiKeeeiqeeuqpits6duwYLVq0iAULFsTWrVt3OMukSZPioIMOii984QtJ38t77w51xhlnRNu2bZOeA4BdRwdX1tg7+De/+U38+c9/3u72t956Ky6++OJ49dVX47DDDou+ffvW+DkB2DV0cGWNvYOPOeaYOO6442L58uVx1VVXVdy+ZcuW+MpXvhKbN2+Or33ta9GiRYsaPycAu4YOrqyxdzAAjYcOrqyxd/Dy5ctj0KBB8cYbb8TXv/71uO6662qchR2xEEVuHXXUUXHaaafFokWLonfv3nHmmWfGmWeeGb169YolS5bEGWeckdls/fv3j86dO8fEiRNjwIABccEFF8RFF10UTz/9dEREXHnllbHbbrvF1VdfHUcddVR87nOfi6OPPjr+5V/+JS6//PKkM0844YS47LLLYuXKlXHkkUfG0KFD45xzzon9998//vrXv8bpp58eEe9v9EZENG/ePCZPnhy9evWKa6+9NvbZZ5847bTT4txzz43BgwdH165d4/jjj4/nnnuuItOyZcsYMmRILF26NPr06RNf+MIX4qKLLoq777674jHl5eUxb968WLRoUdL3cu+990aEj8sDaKh0cGWNvYOnTp0ahx9+eOy///4xbNiw+NznPhcnnXRS9OzZM37xi1/ERz7ykbj//vujpKQk6foAUDw6uLLG3sEREXfffXd06NAhfvKTn8Thhx8eI0aMiAMPPDAefPDB+MQnPhFjx45NujYAFJcOriwPHXzjjTdG//79o3///nHRRRdFxLsfV/vebf37948333wz6foAUDw6uLLG3sFf+tKX4pVXXonWrVvHihUrYtSoUdt9XXHFFUnXhqbJQhS5NmXKlPjWt74VnTp1iv/+7/+OOXPmxIgRI2L27NnRrl27zObafffd45FHHonTTjst5s6dG+PHj4+77rorXn755YiIOPHEE+PJJ5+MU045JV599dV4+OGHo2XLlvG73/0uRo8enXzuj370o7jzzjujd+/e8cQTT8T06dNj0KBBMXv27HjnnXciIqJDhw6VMh/96EfjhRdeiJtuuin23nvvmD17djz44IPx8ssvx8c//vG4/fbb4/Of/3ylzM9//vM477zzYuXKlXHffffFXXfdFTNmzEie+4OeffbZePnll6Nz584xaNCgojwnAMWngytrzB180UUXxVe+8pVo06ZNPPXUU/HAAw/Eiy++GIceemh897vfjb/85S9x8MEH1+kMAIpHB1fWmDv4g7OMGjUq/vnPf8akSZOiWbNmcc0118S0adOitLS0zmcAUBw6uLLG3sELFiyIZ555Jp555pn4y1/+EhERa9asqbjtmWee2e4dNADIhg6urDF38HsfI7h+/fq45557dvg1ceLEOp1B01JSKBQKWQ8BZGvt2rXRq1ev2LBhQ6xatSp22223rEcCgCZBBwNANnQwAGRDBwNANnQwTZF3iIIm5G9/+1usX7++0m2rV6+Oiy++OFasWBEjRoxQfgCwC+hgAMiGDgaAbOhgAMiGDob3eYcoaEIuueSS+PWvfx19+/aNbt26xYoVK+KFF16It956K/bbb7+YPXt2dOrUKesxASB3dDAAZEMHA0A2dDAAZEMHw/uaZz0AUH/OOuusWLp0acyZMyeeffbZiIjo1atXXHTRRXHllVdu93mxAEBx6GAAyIYOBoBs6GAAyIYOhvd5hygAAAAAAAAAACA3mmU9AAAAAAAAAAAAQLE0uI/M27ZtWyxZsiTatGkTJSUlWY8DQCNSKBRizZo10b1792jWzM5vbelgAFLp4LrRwQCk0sF1o4MBSKWD60YHA5CqNh3c4BailixZEj169Mh6DAAascWLF8fee++d9RiNjg4GoK50cBodDEBd6eA0OhiAutLBaXQwAHVVkw5ucCvLbdq0yXoEABo5XZLGdQOgrnRJGtcNgLrSJWlcNwDqSpekcd0AqKuadEmDW4jytogA1JUuSeO6AVBXuiSN6wZAXemSNK4bAHWlS9K4bgDUVU26pMEtRAEAAAAAAAAAAKTaZQtRt99+e+y7776x++67R79+/eLZZ5/dVUcBAB+ggwEgGzoYALKhgwEgGzoYgIZslyxE3X///TFmzJi47rrr4k9/+lP06dMnBg8eHMuXL98VxwEA/58OBoBs6GAAyIYOBoBs6GAAGrqSQqFQKPaT9uvXL44++uj4P//n/0RExLZt26JHjx7x1a9+Na6++upqs6tXr46ysrJijwRAE1JeXh5t27bNeoxM6GAAsqSDdTAA2dDBOhiAbOhgHQxANmrSwUV/h6hNmzbFnDlzYuDAge8f0qxZDBw4MGbNmrXd4zdu3BirV6+u9AUA1J4OBoBs6GAAyIYOBoBs6GAAGoOiL0StWLEitm7dGl26dKl0e5cuXWLp0qXbPX7cuHFRVlZW8dWjR49ijwQATYIOBoBs6GAAyIYOBoBs6GAAGoOiL0TV1tixY6O8vLzia/HixVmPBABNgg4GgGzoYADIhg4GgGzoYACy0LzYT9ixY8fYbbfdYtmyZZVuX7ZsWXTt2nW7x5eWlkZpaWmxxwCAJkcHA0A2dDAAZEMHA0A2dDAAjUHR3yGqZcuW0bdv35g2bVrFbdu2bYtp06bFscceW+zjAID/TwcDQDZ0MABkQwcDQDZ0MACNQdHfISoiYsyYMTFy5Mg46qij4phjjolbb7011q1bF+eff/6uOA4A+P90MABkQwcDQDZ0MABkQwcD0NDtkoWoc845J/75z3/GtddeG0uXLo0jjjgipk6dGl26dNkVxwEA/58OBoBs6GAAyIYOBoBs6GAAGrqSQqFQyHqID1q9enWUlZVlPQYAjVh5eXm0bds26zEaHR0MQF3p4DQ6GIC60sFpdDAAdaWD0+hgAOqqJh3crJ5mAQAAAAAAAAAA2OUsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALnRPOsBAAAovhYtWiTlvv71ryflzjjjjKRcv379knIRESUlJUm56dOnJ+UmT56clIuIuO2225JyhUIh+UwAAACIiLj55puTcmPHjk0+87jjjkvKPf3008lnAgDAB3mHKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQG82zHgAAgOIbMGBAUu673/1ucQfZiUKhUO/ZE088sV5zERFt2rRJyt10003JZwLArlJaWpqU22uvvZJyd911V1IuIuKTn/xkUm7btm1JuYkTJyblIiK+9a1vJeXmz5+ffCYAVKcuf2YHoOm46qqrknIf+9jHknJ33HFHUu4rX/lKUi4ivRMXLlyYlNtjjz2SchERnTp1SsqVlJQk5R588MGkXETE5MmTk7NQE94hCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHKj6AtR119/fZSUlFT6Ouigg4p9DADwIToYALKhgwEgGzoYALKhgwFoDJrviic95JBD4g9/+MP7hzTfJccAAB+igwEgGzoYALKhgwEgGzoYgIZulzRT8+bNo2vXrjV67MaNG2Pjxo0Vv169evWuGAkAmgQdDADZ0MEAkA0dDADZ0MEANHRF/8i8iIhXXnklunfvHvvtt1+ce+65sWjRoiofO27cuCgrK6v46tGjx64YCQCaBB0MANnQwQCQDR0MANnQwQA0dEVfiOrXr1+MHz8+pk6dGnfccUe89tprccIJJ8SaNWt2+PixY8dGeXl5xdfixYuLPRIANAk6GACyoYMBIBs6GACyoYMBaAyK/pF5Q4cOrfjnww8/PPr16xc9e/aM3/72t3HhhRdu9/jS0tIoLS0t9hgA0OToYADIhg4GgGzoYADIhg4GoDHYJR+Z90Ht2rWL3r17x/z583f1UQDAB+hgAMiGDgaAbOhgAMiGDgagIdrlC1Fr166NBQsWRLdu3Xb1UQDAB+hgAMiGDgaAbOhgAMiGDgagISr6QtQVV1wRM2bMiIULF8bTTz8dZ555Zuy2227x2c9+tthHAQAfoIMBIBs6GACyoYMBIBs6GIDGoHmxn/D111+Pz372s7Fy5cro1KlTHH/88TF79uzo1KlTsY8CAD5ABwNANnQwAGRDBwNANnQwAI1BSaFQKGQ9xAetXr06ysrKsh4jlw466KCk3H/9138ln/nYY48l5f793/89Kbdp06akHJAv5eXl0bZt26zHaHR0cL6UlJQk5f7t3/4tKXfkkUcm5S655JKkXBZmzpyZnD3iiCOScr/61a+Scueff35SDupKB6fRwWRhn332Sc7+/Oc/T8qdcsopyWemSv09URY/LnvkkUeScmeffXZSrnXr1km5iIjbbrstKff9738/KffSSy8l5ZoSHZxGB9NU3HzzzUm5q6++OvnM448/Pin39NNPJ58JWdDBaXRww3PyyScnZ++///6kXPv27ZPPTJH658OIbP6MWN9Sr8+aNWuSz0z9OfakSZOSzyQ/atLBRf/IPAAAAAAAAAAAgKxYiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcsBAFAAAAAAAAAADkhoUoAAAAAAAAAAAgNyxEAQAAAAAAAAAAuWEhCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcsBAFAAAAAAAAAADkRvOsB6D2jjrqqKTcAw88kJTbd999k3IREccff3xS7jOf+UxS7oYbbkjKTZ06NSkXEbF58+bkLADsKoVCISl36623FneQBmjo0KFJuQMOOCD5zJKSkqTc4MGDk3J77LFHUi4iYt26dclZAOpfaWlpUu6aa65JPvOUU05JzlK1T33qU0m5iy++OCl33333JeUi0l8D77zzTlLukksuScoBAEBTc+KJJyblUv8eOSKiXbt2ydnG4u23307KPfLII0WeZNfp27dvUu7ggw9OPvPuu+9OyqX+/cfkyZOTcjRe3iEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAORG86wHoPaOO+64pNy+++5b3EF2ob59+yblfv/73yflXnjhhaRcRMTXv/71pNwTTzyRfCYAENG1a9ek3He/+92kXJs2bZJydfH0008n5d55550iTwJAQ/Wzn/0sKff5z3++yJPsOhMnTkzO3nzzzUm5bdu2JeW+9rWvJeUiIi644IKk3DXXXJOUu/3225NyERGPP/54Uq5///5Jub322ispFxHx9ttvJ2cB8mLw4MFZjwBAPfn0pz+dlGvXrl1xB9mFnn/++aTcjTfemHzmrFmzknJvvfVW8pn1LfXn30cddVTymY899lhS7le/+lVSbr/99kvKRUT885//TM6SHe8QBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuNM96ABq+jRs3Jme/8IUvJOUOPvjgpNyoUaOSch//+MeTchERv//975Nyf/vb35Jy3/3ud5NyERFPP/10Uu7QQw9Nyo0cOTIpFxHxi1/8Iik3c+bM5DMBSNeiRYukXOrvFSIifvSjHyXl9txzz+QzU61bty4p99xzzyXltm3blpQDIDsvvvhiUu6QQw5JyhUKhaRcXVx99dVJudTOj6j/Tpw4cWJy9oILLkjKdezYMfnM+pb684Vhw4Yln3n33XcnZwHyYu7cuUm5uvzc/JRTTknKpf4MG4B3DRgwIClXUlJS3EFq4Ne//nVSri5//0jV1qxZk5T7y1/+knzm/Pnzk3If/ehHk3JXXHFFUi4i4qqrrkrOkh3vEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbjTPegBqb+rUqUm5JUuWJOW6d++elIuIuP/++5NyI0eOTMqde+65SbkbbrghKRcRMXDgwKTc0UcfnZSbPHlyUq6xWbRoUVJu5syZRZ4EgJr40pe+lJT7yU9+UuRJdp3nnnsuOZv6e6If//jHyWcCkI2+ffsm5Xr16pWUa9Ys7b91e/3115NyERGXX355Um7ixInJZzYWJ5xwQnK2pKQkKTdjxoykXJs2bZJyERFHHnlkUi71e5wzZ05SDoB3LVu2rN7P7Ny5c72fCUD6n0kLhUKRJ9m5Rx99tN7PpPiWL1+enL3jjjuScj/84Q+Tcp06dUrK0Xh5hygAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIjVovRM2cOTNOP/306N69e5SUlMTkyZMr3V8oFOLaa6+Nbt26RatWrWLgwIHxyiuvFGteAGiydDAAZEMHA0A2dDAAZEMHA5AHtV6IWrduXfTp0yduv/32Hd5/yy23xG233RZ33nlnPPPMM7HHHnvE4MGDY8OGDXUeFgCaMh0MANnQwQCQDR0MANnQwQDkQfPaBoYOHRpDhw7d4X2FQiFuvfXW+Pa3vx1nnHFGRET88pe/jC5dusTkyZNjxIgRdZsWAJowHQwA2dDBAJANHQwA2dDBAORBrd8hqjqvvfZaLF26NAYOHFhxW1lZWfTr1y9mzZq1w8zGjRtj9erVlb4AgNrRwQCQDR0MANnQwQCQDR0MQGNR1IWopUuXRkREly5dKt3epUuXivs+bNy4cVFWVlbx1aNHj2KOBABNgg4GgGzoYADIhg4GgGzoYAAai6IuRKUYO3ZslJeXV3wtXrw465EAoEnQwQCQDR0MANnQwQCQDR0MQBaKuhDVtWvXiIhYtmxZpduXLVtWcd+HlZaWRtu2bSt9AQC1o4MBIBs6GACyoYMBIBs6GIDGoqgLUb169YquXbvGtGnTKm5bvXp1PPPMM3HssccW8ygA4AN0MABkQwcDQDZ0MABkQwcD0Fg0r21g7dq1MX/+/Ipfv/baazF37txo37597LPPPnHZZZfFTTfdFB/96EejV69ecc0110T37t1j2LBhxZwbAJocHQwA2dDBAJANHQwA2dDBAORBrReinn/++Tj55JMrfj1mzJiIiBg5cmSMHz8+rrzyyli3bl1cfPHFsWrVqjj++ONj6tSpsfvuuxdvagBognQwAGRDBwNANnQwAGRDBwOQB7VeiBowYEAUCoUq7y8pKYnvfOc78Z3vfKdOgwEAlelgAMiGDgaAbOhgAMiGDgYgD2q9EEX25s2bl5Q75JBDknIjRoxIykVErF+/Pik3derUpNzy5cuTckOHDk3KRUR06tQpKXfllVcm5c4666ykXEREaWlpUq5Lly7JZwLQNMyePTvrEXa50aNHJ2fnzJlTxEkA2NXatGmTnP3Wt76VlGvVqlVS7pVXXknKXX755Um5iIhHHnkkOZt3CxcuTM4uWbIkKXfttdcm5fbYY4+kXETEwQcfnJSr7i/VqrNt27akHAAANDUTJ05Myg0ePDj5zAkTJiTlHn300eQzyYe//e1v9Xpe69at6/U8stcs6wEAAAAAAAAAAACKxUIUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDeaZz0A9WfVqlVJuTvvvLO4gzRAW7ZsSc6++eabSbnLL7+8XnMREXvttVdS7sUXX0zK7b333kk5ABqfF154ISl3yCGHJJ950003JeXOPPPMpNz111+flIuI+O1vf5uUmzBhQlKuLr+3ASDi2GOPTc5+5jOfKeIkO/fNb34zKffII48UeRIiIu66667k7LRp05JyCxcuTMp17do1KQcANfHggw9mPQJAk3TZZZcl5fbYY4/kM+fPn5+cpWlL/fPsihUrknL/+Mc/knI0Xt4hCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcsBAFAAAAAAAAAADkhoUoAAAAAAAAAAAgNyxEAQAAAAAAAAAAuWEhCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcaJ71AED96dmzZ1KuY8eORZ4EgLzZunVrUu7vf/978pnnnHNOUu4Xv/hFUu6ss85KykVEfPKTn0zKXXvttUm5W265JSkXEfHb3/42KVdeXp58JkBDc95559X7mffcc09SbuLEiUWehKwsXLiwXs/7whe+UK/nRUTMmjUrKbdgwYIiTwLArrZp06asRwBokt58882sR4AaGz58eFLO311TU94hCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcsBAFAAAAAAAAAADkhoUoAAAAAAAAAAAgNyxEAQAAAAAAAAAAuWEhCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcaJ71AED9KRQK9ZoDgF1p69atSbmRI0cm5caNG5eUi4g45phjknKnnXZaUu7OO+9MykVEnH322Um5b37zm0m5OXPmJOUAaiL1//OHDx9e5El27sYbb6z3M2naOnbsWO9nrl27Nin3zjvvFHkSAGrijTfeSM7OnTu3eIMAAA1Wnz59krM33XRTESeB7XmHKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQG82zHgAauyuuuCIpt3Tp0qTc1q1bk3IREa1bt07KtWrVKvnMVN26dUvKlZWVJeXKy8uTcgA0HX//+9/rPfvrX/86KXfccccl5SIiBg4cmJT7xCc+kZT7zGc+k5SLiHjiiSeSs0DTcOqppyblSktLizzJzr3zzjv1fiZN2ze+8Y3k7LZt25JyM2fOTD4TgPq3efPm5Oz69euLOAkA0FB16tQpOVsoFIo4yc6tWLGiXs8je94hCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHKj1gtRM2fOjNNPPz26d+8eJSUlMXny5Er3jxo1KkpKSip9DRkypFjzAkCTpYMBIBs6GACyoYMBIBs6GIA8qPVC1Lp166JPnz5x++23V/mYIUOGxJtvvlnxNWHChDoNCQDoYADIig4GgGzoYADIhg4GIA+a1zYwdOjQGDp0aLWPKS0tja5du9bo+TZu3BgbN26s+PXq1atrOxIANAk6GACyoYMBIBs6GACyoYMByINav0NUTUyfPj06d+4cBx54YHz5y1+OlStXVvnYcePGRVlZWcVXjx49dsVIANAk6GAAyIYOBoBs6GAAyIYOBqChK/pC1JAhQ+KXv/xlTJs2Lb73ve/FjBkzYujQobF169YdPn7s2LFRXl5e8bV48eJijwQATYIOBoBs6GAAyIYOBoBs6GAAGoNaf2TezowYMaLinw877LA4/PDDY//994/p06fHqaeeut3jS0tLo7S0tNhjAECTo4MBIBs6GACyoYMBIBs6GIDGYJd8ZN4H7bffftGxY8eYP3/+rj4KAPgAHQwA2dDBAJANHQwA2dDBADREu3wh6vXXX4+VK1dGt27ddvVRAMAH6GAAyIYOBoBs6GAAyIYOBqAhqvVH5q1du7bSdu9rr70Wc+fOjfbt20f79u3jhhtuiOHDh0fXrl1jwYIFceWVV8YBBxwQgwcPLurgANDU6GAAyIYOBoBs6GAAyIYOBiAPar0Q9fzzz8fJJ59c8esxY8ZERMTIkSPjjjvuiBdffDHuueeeWLVqVXTv3j0GDRoUN954o8+FBYA60sEAkA0dDADZ0MEAkA0dDEAelBQKhULWQ3zQ6tWro6ysLOsxaGLatWuXnJ09e3ZS7sADD0w+k6q9+uqrSbny8vLkM+fNm5eUO++885JyW7ZsSco1JeXl5dG2bdusx2h0dDDkx7hx45KzV155ZREn2blp06YlZwcNGlTESSgGHZxGB+8699xzT1Lu3HPPTT7zqaeeSsql/pfUGzZsSMqRHyNHjkzK/eIXv0g+c/369Um50047LSmX+rOXpkQHp9HBNBUTJkxIyvXr1y/5zP322y85C42JDk6jg4mI6N69e1Iu9c9AAwYMSMpFRDSwlYpdoqSkJCn3sY99LPnM1I/ZTP2Z8umnn56Ui4jYtGlTcpZdoyYd3KyeZgEAAAAAAAAAANjlLEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcqN51gNAQ7Bq1ark7EEHHZSUu/zyy5NyZ511VlIuIuL4449PzjYW++23X1Luz3/+c/KZqf8ut2zZknwmAFTnggsuqPczt23blpRbtmxZkScByNYTTzyRlNuwYUORJ6Gx6d69e1Luhz/8YZEn2bnHHnssKTd79uwiTwJATfTv3z8pVygUijwJALzvnnvuScqdfPLJSbmSkpKkXETT6MTU65PFtenbt29S7vzzz08+82c/+1lylux4hygAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBvNsx4Amqof//jHSblbb701+cxJkyYl5c4444zkM1N9+ctfTspNnz49KffPf/4zKRcRsXLlyuQsAFRnwIABSbl27doVdY6a+NWvfpWUu+CCC4o8CZBHrVq1Ssr16tWryJPs3D/+8Y96P5OGo3v37snZb3/720m51N7fsGFDUi4i4gc/+EFyFoB0qf+fv/vuuyfl3nnnnaQcAE3Lvvvum5Q76KCDijsITUbq74luv/325DP79euXlPPz72x5hygAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcqN51gMAtVMoFJKzf/nLX5JyZ5xxRvKZqR599NGk3KJFi4o8CUB22rVrl5zdvHlzUm7dunXJZ+Zdq1atkrMjRoxIyv3gBz9IyjVvXv+/zZ88eXK9nwk0HWVlZUm5T3ziE0WehKaic+fOSbmHHnoo+cw+ffokZ1OcffbZydmnn366iJMAUFOHHHJIUq5Lly5JuYULFyblAGhaevfunZTr3r17kSep3o033picnThxYlLu97//fVKuZ8+eSbm6aNYs7b10Nm3alHzmypUrk3IdOnRIytXl5+bnn39+Um7kyJFJudS/G4iIuPnmm5Ny5eXlyWc2VN4hCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcsBAFAAAAAAAAAADkhoUoAAAAAAAAAAAgNyxEAQAAAAAAAAAAuWEhCgAAAAAAAAAAyA0LUQAAAAAAAAAAQG5YiAIAAAAAAAAAAHLDQhQAAAAAAAAAAJAbFqIAAAAAAAAAAIDcaJ71AED9ee2117IeAYBaeOCBB5Kz8+fPT8p9+ctfTj6zvjVrlrbbf8oppyTlxowZk5SLiBg8eHByNsWGDRuSs9/61reSco899ljymQA7s23btqTcxo0bk3K77757Ui4iYsiQIUm5X/7yl0m5rVu3JuWaikMPPTQp97//+79FnmTX+cEPfpCU++///u8iTwIAADRFq1evTsqtX78+KdeqVauk3DXXXJOUi4j49re/nZxNUSgU6vW8iIhXXnklKXf55Zcnn/nII48k5T71qU8l5a6++uqkXETERz/60aRcx44dk3Jf//rXk3IREWeddVZS7rbbbkvK/fSnP03K1QfvEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5UauFqHHjxsXRRx8dbdq0ic6dO8ewYcNi3rx5lR6zYcOGGD16dHTo0CH23HPPGD58eCxbtqyoQwNAU6ODASAbOhgAsqGDASAbOhiAvKjVQtSMGTNi9OjRMXv27Hjsscdi8+bNMWjQoFi3bl3FYy6//PJ46KGH4oEHHogZM2bEkiVL4qyzzir64ADQlOhgAMiGDgaAbOhgAMiGDgYgL5rX5sFTp06t9Ovx48dH586dY86cOXHiiSdGeXl53HXXXXHffffFKaecEhERd999dxx88MExe/bs6N+/f/EmB4AmRAcDQDZ0MABkQwcDQDZ0MAB5Uat3iPqw8vLyiIho3759RETMmTMnNm/eHAMHDqx4zEEHHRT77LNPzJo1a4fPsXHjxli9enWlLwCgejoYALKhgwEgGzoYALKhgwForJIXorZt2xaXXXZZHHfccXHooYdGRMTSpUujZcuW0a5du0qP7dKlSyxdunSHzzNu3LgoKyur+OrRo0fqSADQJOhgAMiGDgaAbOhgAMiGDgagMUteiBo9enS89NJL8Zvf/KZOA4wdOzbKy8srvhYvXlyn5wOAvNPBAJANHQwA2dDBAJANHQxAY9Y8JXTppZfGww8/HDNnzoy999674vauXbvGpk2bYtWqVZW2gpctWxZdu3bd4XOVlpZGaWlpyhgA0OToYADIhg4GgGzoYADIhg4GoLGr1TtEFQqFuPTSS2PSpEnx+OOPR69evSrd37dv32jRokVMmzat4rZ58+bFokWL4thjjy3OxADQBOlgAMiGDgaAbOhgAMiGDgYgL2r1DlGjR4+O++67L6ZMmRJt2rSp+BzYsrKyaNWqVZSVlcWFF14YY8aMifbt20fbtm3jq1/9ahx77LHRv3//XfINAEBToIMBIBs6GACyoYMBIBs6GIC8qNVC1B133BEREQMGDKh0+9133x2jRo2KiIgf//jH0axZsxg+fHhs3LgxBg8eHP/xH/9RlGEBoKnSwQCQDR0MANnQwQCQDR0MQF7UaiGqUCjs9DG777573H777XH77bcnDwUAVKaDASAbOhgAsqGDASAbOhiAvKjVQhTQuP3973/PegQAamHLli3J2QsvvDApt3nz5qTcP/7xj6RcXZxzzjlJub59+xZ5kobn0UcfTc7eeuutxRsEoEiWL1+elDv33HOTcuPHj0/KRUQMHz48KXffffcl5W6++eakXBbOO++8pFynTp2Sz/z85z+flKvJXwIV28svv5yUe+ihh4o8CQANVX3/eW3q1Kn1eh4AjdPs2bOTct/73veSctdff31Srqn4wQ9+kJS78847k3ILFy5MytXFI488Uq+5iIgjjjgiKTdu3LikXF3+HqNXr15Jucsuuywp99Of/jQpVx+aZT0AAAAAAAAAAABAsViIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAORG86wHAABgxy655JLk7NVXX52UGz16dPKZVG39+vVJud/97ndJuYsvvjgpB5A3kydPTsrdf//9yWdeeOGFSbnhw4cn5c4+++ykXKFQSMo1FWvXrk3KzZo1K/nML3/5y0m5hQsXJp8JQOOy33771et5y5cvr9fzAGhabr/99qTcG2+8kZQbMGBAUi4i/c/QqX9emzhxYlIuIuLvf/97Um7Lli3JZzYFc+fOTcoNHTo0Kbfvvvsm5SIi9txzz6TcypUrk89sqLxDFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5YSEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAOSGhSgAAAAAAAAAACA3LEQBAAAAAAAAAAC5UVIoFApZD/FBq1evjrKysqzHgFw6/vjjk3J//OMfizzJzvXs2TMpt2jRoiJPQmNUXl4ebdu2zXqMRkcH50uLFi2ScoMHD07KDRs2LCnXr1+/pFxExF/+8pekXKdOnZJyU6ZMScpFRPzf//t/k3Lz5s1LPhOyoIPT6OCGp02bNsnZT33qU0m5IUOGJOXOO++8pFwD+3HQLnPvvfcm5R588MGkXF1+vwB1oYPT6GAamx/96EdJuVGjRiXl+vTpk5SLiFi8eHFyFhoTHZxGBwNQVzXpYO8QBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByo6RQKBSyHuKDVq9eHWVlZVmPAbnUvXv3pNwLL7yQlOvcuXNSLiKiZ8+eSblFixYln0l+lJeXR9u2bbMeo9HRwQDUlQ5Oo4Opiy5duiTlhg4dmnzmkCFDknLr1q1Lyk2dOjUpFxHxwAMPJGehMdHBaXQwAHWlg9PoYADqqiYd7B2iAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMiN5lkPANSfJUuWJOVmz56dlFu1alVSLiJi2bJlyVkAAICmIvXPTuPHj08+sy5ZAAAAAKgP3iEKAAAAAAAAAADIDQtRAAAAAAAAAABAbliIAgAAAAAAAAAAcqNWC1Hjxo2Lo48+Otq0aROdO3eOYcOGxbx58yo9ZsCAAVFSUlLp65JLLinq0ADQ1OhgAMiGDgaAbOhgAMiGDgYgL2q1EDVjxowYPXp0zJ49Ox577LHYvHlzDBo0KNatW1fpcV/84hfjzTffrPi65ZZbijo0ADQ1OhgAsqGDASAbOhgAsqGDAciL5rV58NSpUyv9evz48dG5c+eYM2dOnHjiiRW3t27dOrp27Vqj59y4cWNs3Lix4terV6+uzUgA0CToYADIhg4GgGzoYADIhg4GIC9q9Q5RH1ZeXh4REe3bt690+7333hsdO3aMQw89NMaOHRvr16+v8jnGjRsXZWVlFV89evSoy0gA0CToYADIhg4GgGzoYADIhg4GoLEqKRQKhZTgtm3b4jOf+UysWrUqnnzyyYrb//M//zN69uwZ3bt3jxdffDGuuuqqOOaYY+LBBx/c4fPsaCNYCULDMmXKlKTcqlWrks+8+OKLk3If/P8Tmq7y8vJo27Zt1mPsMjoYgIZKB+tgALKhg3UwANnQwToYgGzUpINr9ZF5HzR69Oh46aWXKpVfROUlhsMOOyy6desWp556aixYsCD233//7Z6ntLQ0SktLU8cAgCZHBwNANnQwAGRDBwNANnQwAI1Z0kfmXXrppfHwww/HE088EXvvvXe1j+3Xr19ERMyfPz/lKADgA3QwAGRDBwNANnQwAGRDBwPQ2NXqHaIKhUJ89atfjUmTJsX06dOjV69eO83MnTs3IiK6deuWNCAAoIMBICs6GACyoYMBIBs6GIC8qNVC1OjRo+O+++6LKVOmRJs2bWLp0qUREVFWVhatWrWKBQsWxH333Ref/OQno0OHDvHiiy/G5ZdfHieeeGIcfvjhu+QbAICmQAcDQDZ0MABkQwcDQDZ0MAB5UVIoFAo1fnBJyQ5vv/vuu2PUqFGxePHi+PznPx8vvfRSrFu3Lnr06BFnnnlmfPvb3462bdvW6IzVq1dHWVlZTUcC6sGUKVOScqtWrUo+84OfP10bGzduTD6T/CgvL69x7zQWOhiAxkAH62AAsqGDdTAA2dDBOhiAbNSkg2u1EFUfFCAAdZXHP4TWBx0MQF3p4DQ6GIC60sFpdDAAdaWD0+hgAOqqJh3crJ5mAQAAAAAAAAAA2OUsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMiNBrcQVSgUsh4BgEZOl6Rx3QCoK12SxnUDoK50SRrXDYC60iVpXDcA6qomXdLgFqLWrFmT9QgANHK6JI3rBkBd6ZI0rhsAdaVL0rhuANSVLknjugFQVzXpkpJCA1vB3bZtWyxZsiTatGkTJSUl292/evXq6NGjRyxevDjatm2bwYQNl2tTPdenaq5N9VyfqjW0a1MoFGLNmjXRvXv3aNaswe38Nng6OJ1rUz3Xp2quTfVcn6o1tGujg+tGB6dzbarn+lTNtame61O1hnZtdHDd6OB0rk31XJ+quTbVc32q15Cujw6uGx2czrWpnutTNdemeq5P9RrS9alNBzevp5lqrFmzZrH33nvv9HFt27bN/EI3VK5N9Vyfqrk21XN9qtaQrk1ZWVnWIzRaOrjuXJvquT5Vc22q5/pUrSFdGx2cTgfXnWtTPdenaq5N9VyfqjWka6OD0+ngunNtquf6VM21qZ7rU72Gcn10cDodXHeuTfVcn6q5NtVzfarXUK5PTTvYyjIAAAAAAAAAAJAbFqIAAAAAAAAAAIDcaHQLUaWlpXHddddFaWlp1qM0OK5N9Vyfqrk21XN9qubaNC3+fVfNtame61M116Z6rk/VXJumxb/vqrk21XN9qubaVM/1qZpr07T4910116Z6rk/VXJvquT7Vc32aDv+uq+baVM/1qZprUz3Xp3qN9fqUFAqFQtZDAAAAAAAAAAAAFEOje4coAAAAAAAAAACAqliIAgAAAAAAAAAAcsNCFAAAAAAAAAAAkBsWogAAAAAAAAAAgNywEAUAAAAAAAAAAORGo1qIuv3222PfffeN3XffPfr16xfPPvts1iM1CNdff32UlJRU+jrooIOyHisTM2fOjNNPPz26d+8eJSUlMXny5Er3FwqFuPbaa6Nbt27RqlWrGDhwYLzyyivZDJuBnV2fUaNGbfdaGjJkSDbD1rNx48bF0UcfHW3atInOnTvHsGHDYt68eZUes2HDhhg9enR06NAh9txzzxg+fHgsW7Yso4nrV02uz4ABA7Z7/VxyySUZTUyx6eAd08Hv08HV08FV08HV08Ho4B3Twe/TwdXTwVXTwdXTwejgHdPBlenhqungqung6ulgdPCO6eDKdHDVdHDVdHD18tjBjWYh6v77748xY8bEddddF3/605+iT58+MXjw4Fi+fHnWozUIhxxySLz55psVX08++WTWI2Vi3bp10adPn7j99tt3eP8tt9wSt912W9x5553xzDPPxB577BGDBw+ODRs21POk2djZ9YmIGDJkSKXX0oQJE+pxwuzMmDEjRo8eHbNnz47HHnssNm/eHIMGDYp169ZVPObyyy+Phx56KB544IGYMWNGLFmyJM4666wMp64/Nbk+ERFf/OIXK71+brnllowmpph0cPV08Lt0cPV0cNV0cPV0cNOmg6ung9+lg6ung6umg6ung5s2HVw9Hfw+PVw1HVw1HVw9Hdy06eDq6eD36eCq6eCq6eDq5bKDC43EMcccUxg9enTFr7du3Vro3r17Ydy4cRlO1TBcd911hT59+mQ9RoMTEYVJkyZV/Hrbtm2Frl27Fr7//e9X3LZq1apCaWlpYcKECRlMmK0PX59CoVAYOXJk4YwzzshknoZm+fLlhYgozJgxo1AovPtaadGiReGBBx6oeMzf/va3QkQUZs2aldWYmfnw9SkUCoWTTjqp8G//9m/ZDcUuo4OrpoN3TAdXTwdXTwdXTwc3LTq4ajp4x3Rw9XRw9XRw9XRw06KDq6aDq6aHq6aDq6eDq6eDmxYdXDUdXDUdXDUdXD0dXL08dHCjeIeoTZs2xZw5c2LgwIEVtzVr1iwGDhwYs2bNynCyhuOVV16J7t27x3777RfnnntuLFq0KOuRGpzXXnstli5dWul1VFZWFv369fM6+oDp06dH586d48ADD4wvf/nLsXLlyqxHykR5eXlERLRv3z4iIubMmRObN2+u9Po56KCDYp999mmSr58PX5/33HvvvdGxY8c49NBDY+zYsbF+/fosxqOIdPDO6eCd08E1o4PfpYOrp4ObDh28czp453Rwzejgd+ng6ungpkMH75wOrhk9vHM6+F06uHo6uOnQwTung2tGB++cDn6XDq5eHjq4edYD1MSKFSti69at0aVLl0q3d+nSJf7+979nNFXD0a9fvxg/fnwceOCB8eabb8YNN9wQJ5xwQrz00kvRpk2brMdrMJYuXRoRscPX0Xv3NXVDhgyJs846K3r16hULFiyIb37zmzF06NCYNWtW7LbbblmPV2+2bdsWl112WRx33HFx6KGHRsS7r5+WLVtGu3btKj22Kb5+dnR9IiI+97nPRc+ePaN79+7x4osvxlVXXRXz5s2LBx98MMNpqSsdXD0dXDM6eOd08Lt0cPV0cNOig6ung2tGB++cDn6XDq6eDm5adHD1dHDN6eHq6eB36eDq6eCmRQdXTwfXnA6ung5+lw6uXl46uFEsRFG9oUOHVvzz4YcfHv369YuePXvGb3/727jwwgsznIzGZsSIERX/fNhhh8Xhhx8e+++/f0yfPj1OPfXUDCerX6NHj46XXnqpSX/2cnWquj4XX3xxxT8fdthh0a1btzj11FNjwYIFsf/++9f3mFAvdDDFooPfpYOrp4PhfTqYYtHB79LB1dPB8D4dTLHo4Hfp4OrpYHifDqZYdPC7dHD18tLBjeIj8zp27Bi77bZbLFu2rNLty5Yti65du2Y0VcPVrl276N27d8yfPz/rURqU914rXkc1t99++0XHjh2b1Gvp0ksvjYcffjieeOKJ2HvvvStu79q1a2zatClWrVpV6fFN7fVT1fXZkX79+kVENKnXTx7p4NrRwTumg2tPB+vgD9PBTY8Orh0dvGM6uPZ0sA7+MB3c9Ojg2tHBVdPDtaODdfCH6eCmRwfXjg6umg6uHR2sgz8sTx3cKBaiWrZsGX379o1p06ZV3LZt27aYNm1aHHvssRlO1jCtXbs2FixYEN26dct6lAalV69e0bVr10qvo9WrV8czzzzjdVSF119/PVauXNkkXkuFQiEuvfTSmDRpUjz++OPRq1evSvf37ds3WrRoUen1M2/evFi0aFGTeP3s7PrsyNy5cyMimsTrJ890cO3o4B3TwbWng9+ng3VwU6WDa0cH75gOrj0d/D4drIObKh1cOzq4anq4dnTw+3SwDm6qdHDt6OCq6eDa0cHv08H56+BG85F5Y8aMiZEjR8ZRRx0VxxxzTNx6662xbt26OP/887MeLXNXXHFFnH766dGzZ89YsmRJXHfddbHbbrvFZz/72axHq3dr166ttH342muvxdy5c6N9+/axzz77xGWXXRY33XRTfPSjH41evXrFNddcE927d49hw4ZlN3Q9qu76tG/fPm644YYYPnx4dO3aNRYsWBBXXnllHHDAATF48OAMp64fo0ePjvvuuy+mTJkSbdq0qfgc2LKysmjVqlWUlZXFhRdeGGPGjIn27dtH27Zt46tf/Woce+yx0b9//4yn3/V2dn0WLFgQ9913X3zyk5+MDh06xIsvvhiXX355nHjiiXH44YdnPD11pYOrpoPfp4Orp4OrpoOrp4ObNh1cNR38Ph1cPR1cNR1cPR3ctOngqungyvRw1XRw1XRw9XRw06aDq6aDK9PBVdPBVdPB1ctlBxcakZ/+9KeFffbZp9CyZcvCMcccU5g9e3bWIzUI55xzTqFbt26Fli1bFj7ykY8UzjnnnML8+fOzHisTTzzxRCEitvsaOXJkoVAoFLZt21a45pprCl26dCmUlpYWTj311MK8efOyHboeVXd91q9fXxg0aFChU6dOhRYtWhR69uxZ+OIXv1hYunRp1mPXix1dl4go3H333RWPeeeddwpf+cpXCnvttVehdevWhTPPPLPw5ptvZjd0PdrZ9Vm0aFHhxBNPLLRv375QWlpaOOCAAwrf+MY3CuXl5dkOTtHo4B3Twe/TwdXTwVXTwdXTwejgHdPB79PB1dPBVdPB1dPB6OAd08GV6eGq6eCq6eDq6WB08I7p4Mp0cNV0cNV0cPXy2MElhUKhsKNFKQAAAAAAAAAAgMamWdYDAAAAAAAAAAAAFIuFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALlhIQoAAAAAAAAAAMgNC1EAAAAAAAAAAEBuWIgCAAAAAAAAAAByw0IUAAAAAAAAAACQGxaiAAAAAAAAAACA3LAQBQAAAAAAAAAA5IaFKAAAAAAAAAAAIDcsRAEAAAAAAAAAALnx/wDINQl/uAxMcQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 3000x3000 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_samples(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instantiate the model\n",
    "\n",
    "This step is easy to achieve as we use a built-in Fully Connected Neural Network. Only a few input parameters are needed:\n",
    "- `module__n_layers`: number of Fully Connected layers to use in the model\n",
    "- `module__n_w_bits` and `module__n_a_bits`: respectively the number of bits to use for quantizing the weight and input/activation values as the FHE can currently only compute integers. These numbers should not become too large as it can cause the compilation step to fail (see Compile section below)\n",
    "- `module__n_accum_bits`: The maximal allowed bit-width to target for intermediate accumulators. It is currently set to 15 as the actual maximum bit-width reached during compilation can be up to one bit higher than this targeted value, in this case 16, which is the maximal value that Concrete ML currently supports.\n",
    "- `module__n_hidden_neurons_multiplier`: A factor that is multiplied by the maximal number of active (non-zero weight) neurons for every layer. Default to 4 but set to 1 here in order to speed up all executions without changing the test accuracy by much. More detail in the qqn documentation. \n",
    "- `module__activation_function`: The activation function to use\n",
    "- `max_epochs`: The number of epochs to consider"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"module__n_layers\": 2,\n",
    "    \"module__n_w_bits\": 4,\n",
    "    \"module__n_a_bits\": 4,\n",
    "    \"module__n_hidden_neurons_multiplier\": 0.5,\n",
    "    \"module__activation_function\": nn.ReLU,\n",
    "    \"max_epochs\": 7,\n",
    "}\n",
    "\n",
    "model = NeuralNetClassifier(**params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model\n",
    "\n",
    "The fit method handles Pandas DataFrame as inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  epoch    train_loss    valid_acc    valid_loss     dur\n",
      "-------  ------------  -----------  ------------  ------\n",
      "      1        \u001b[36m0.4004\u001b[0m       \u001b[32m0.9316\u001b[0m        \u001b[35m0.2297\u001b[0m  4.8741\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      2        \u001b[36m0.2072\u001b[0m       \u001b[32m0.9393\u001b[0m        \u001b[35m0.2139\u001b[0m  3.9925\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      3        \u001b[36m0.1842\u001b[0m       \u001b[32m0.9448\u001b[0m        \u001b[35m0.1985\u001b[0m  3.9748\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      4        0.1972       0.9308        0.2337  4.1288\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      5        0.1864       \u001b[32m0.9458\u001b[0m        0.2055  4.0457\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      6        \u001b[36m0.1737\u001b[0m       \u001b[32m0.9511\u001b[0m        0.2048  4.0543\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      7        \u001b[36m0.1628\u001b[0m       0.9412        0.2293  4.0349\n"
     ]
    }
   ],
   "source": [
    "model.fit(X=x_train, y=y_train);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute predictions in the clear\n",
    "\n",
    "We then compute the accuracy score reached by the model when executed in the clear. It is important to understand that no FHE computations are done here. This step is not necessary but helps illustrate what results should we expect from the model. It is therefore used to demonstrate that FHE computations are exact, meaning the FHE accuracy score will exactly match this very one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The test accuracy of the clear model is 0.94\n"
     ]
    }
   ],
   "source": [
    "y_preds_clear = model.predict(x_test, fhe=\"disable\")\n",
    "\n",
    "print(f\"The test accuracy of the clear model is {accuracy_score(y_test, y_preds_clear):.2f}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compile the model\n",
    "\n",
    "A Concrete ML model needs to be compiled on an input-set, usually the train set or one of its sub-set, before being able to predict. This step creates an FHE circuit, which essentially saves elements found in the model's inference (graph of operations, shapes, bit-width precisions, etc.) needed for the compiler when executing the predictions in FHE during the `predict` method. \n",
    "\n",
    "The maximum bit-width that can be reached by any values (inputs, weights, accumulators) in this circuit is currently 16-bits. If this limit is exceeded, the compilation fails and the user needs to change some of the model's parameters (e.g., decrease the number of quantization bits or decrease `module__n_accum_bits`). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Circuit of 12-bits (FHE simulation)\n"
     ]
    }
   ],
   "source": [
    "# Reduce the input-set's length to make the compilation time faster\n",
    "# The input-set should be large enough to be representative of the input data\n",
    "inputset = x_train.head(1000)\n",
    "simulated_fhe_circuit = model.compile(inputset, device=device)\n",
    "\n",
    "# Print the circuit's maximum bit-width reached during compilation\n",
    "print(f\"Circuit of {simulated_fhe_circuit.graph.maximum_integer_bit_width()}-bits (FHE simulation)\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute the accuracy score with FHE simulation\n",
    "\n",
    "Now, we compute the accuracy score reached by the FHE model with FHE simulation. The accuracy score obtained by simulation, which is faster, is expected to be the same as the one obtained in FHE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The test accuracy (with FHE simulation) of the FHE model is 0.94\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model using simulation\n",
    "y_preds_simulated = model.predict(x_test, fhe=\"simulate\")\n",
    "\n",
    "print(\n",
    "    \"The test accuracy (with FHE simulation) of the FHE model is \"\n",
    "    f\"{accuracy_score(y_test, y_preds_simulated):.2f}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Measure FHE inference time\n",
    "\n",
    "\n",
    "Now, let's compute some predictions in FHE. In order to make the computations faster, we will consider a sample of the original test set and compare these results to the expected values.\n",
    "\n",
    "We execute the key generation separately from the predictions in order to be able to measure its execution time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FHE circuit of 12-bits\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Key generation time: 1.85 seconds\n"
     ]
    }
   ],
   "source": [
    "# Print the circuit's maximum bit-width reached during compilation\n",
    "print(f\"FHE circuit of {model.fhe_circuit.graph.maximum_integer_bit_width()}-bits\")\n",
    "\n",
    "time_begin = time.time()\n",
    "model.fhe_circuit.client.keygen(force=True)\n",
    "print(f\"Key generation time: {time.time() - time_begin:.2f} seconds\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Execution time in FHE: 3.51 seconds per sample\n",
      "\n",
      "Expected values: [0, 4, 1]\n",
      "Simulated prediction values: [0 4 1]\n",
      "FHE prediction values: [0 4 1]\n"
     ]
    }
   ],
   "source": [
    "# Reduce the test set\n",
    "n_samples = 3\n",
    "x_test_sample = x_test.head(n_samples)\n",
    "y_test_sample = y_test.head(n_samples)\n",
    "\n",
    "# Execute the predictions using FHE simulation on a few samples\n",
    "simulated_fhe_predictions = model.predict(x_test_sample, fhe=\"simulate\")\n",
    "\n",
    "time_begin = time.time()\n",
    "fhe_predictions = model.predict(x_test_sample, fhe=\"execute\")\n",
    "seconds_per_sample = (time.time() - time_begin) / len(x_test_sample)\n",
    "print(f\"Execution time in FHE: {seconds_per_sample:.2f} seconds per sample\\n\")\n",
    "\n",
    "print(\"Expected values:\", y_test_sample.tolist())\n",
    "print(\"Simulated prediction values:\", simulated_fhe_predictions)\n",
    "print(\"FHE prediction values:\", fhe_predictions)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "In this notebook, we showed how to use a built-in Fully Connected Neural Network classifier on the MNIST data-set using the Concrete ML library in order to make its inference completely secure.\n",
    "\n",
    "Training, compiling and evaluation such a model is intuitive as our API follows most common Machine Learning APIs. In fact, only a few additional parameters related to quantization are requested, such as `module__n_w_bits`, `module__n_a_bits` or `module__n_accum_bits`. Thanks to the internal implementation of Quantize Aware Training (QAT) techniques, the Concrete ML `NeuralNetCLassifier` model reached a high accuracy score.\n",
    "\n",
    "A single FHE execution using the initial model takes several minutes on a multi-core machine. "
   ]
  }
 ],
 "metadata": {
  "execution": {
   "timeout": 10800
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
